Ultra-wide Baseline Aerial Imagery Matching in Urban Environments

Correspondence matching is a fundamental problem in computer vision, having many uses in structure from motion, stereo vision, image registration, pose-estimation, and others. Today, a large amount of aerial imagery is available online via mapping services such as Google Maps. If we were to pick any pair of images from two different aerial views (see Figure 1 for an example), and perform SIFT-based [3] correspondence matching, we would find ourselves with a large number of mismatches due to the large distortions between the images. Even when augmenting these methods with typical robust approaches such as RANSAC [1] and its variants, we would still fail at finding correct correspondences since RANSAC has difficulty calculating the correct model without a large ratio of correct matches to outliers. These difficulties – large distortions, and low ratio of correct matches to outliers – together render traditional methods ineffective. In this paper, we consider the problem of correspondence matching for aerial imagery in urban environments. Our approach builds on multiple ideas in the literature. Namely, A-SIFT [7], patch-based methods [6], Generalized RANSAC framework [8], self-similarity [5], graphbased image matching [2], and geometric-invariance [4]. The main idea behind this work is to combine view-synthesis with multiple point correspondences under a RANSAC-based scheme. Robust model estimation is supported by self-similarity principles and graph-based modeling that drives the sampling process in a restricted manner that allows the correct model to be extracted. Each of these ideas was chosen to deal with specific problems that cause failures in the earlier approaches as we will now briefly describe. An overview of our pipeline is shown in Figure 2. We begin by extracting square patches around detected Harris corners, which we then describe using the Histogram of Oriented Gradients. Our model assumes affine distortions, which leads us to synthesizing transformations to account for probable changes between the two images. We apply our transformations to one of the input pairs only, as follows: Scale = Sx 0 0 0 Sy 0 0 0 1 Shear =  1 Shx 0 Shy 1 0 0 0 1  Rotation = cos(θ) −sin(θ) 0 sin(θ) cos(θ) 0 0 0 1 